1. Field of the Invention
The present invention relates to an image processing technique of generating an image in which defocusing of a region other than a main object is emphasized.
2. Description of the Related Art
One of the techniques of photo shooting is a method of making a depth of field shallow to narrow a focusing range, and blurring the background other than a focused object image. In order to shoot a photo with a shallow depth of field, the stop of a camera needs to be opened to make the focal length long. Generally, a photo can be shot using such technology in a single lens reflex camera in which the photographing lens has a large aperture and various combinations of the stop and the shutter speed can be selected.
In a compact-size digital still camera and a mobile telephone with a camera having an image capturing function that are recently in widespread use, a photographing lens having a small lens aperture and a very short focal length is adopted for miniaturization, and hence the depth of field is very deep. In such devices, therefore, it is difficult to shoot a photo in which the background is blurred using the above technique.
An image processing method has been proposed for generating an image in which the defocusing of the region other than the main object is emphasized more than the defocusing of the image captured by the actual driving of the stop and the lens in order to have the main object such as a person stand out more than the other objects such as the background. This method is hereinafter referred to as background blurring.
In order to realize background blurring, the process of separating the region of the main object such as the person from the region of other objects such as the background is necessary. One of such processing methods is a method for capturing two images with different in-focus positions, and separating the region of the main object and the region of other objects by increasing/decreasing the edge amount between the images.
If the image capturing is carried out with different in-focus positions when the camera is held in the photographer's hands, each captured image has relative displacement caused by hand movement and the like. Thus, in order to apply the method described above, the increase/decrease of the edge amount needs to be calculated after correcting the relative displacement and aligning the images.
However, when calculating the displacement amount for the alignment, the accuracy in the result of the displacement amount calculation may decrease since a correlation calculation is performed between the images having different defocus degrees.
As a method for realizing background blurring after aligning the images having different in-focus positions, for example, Japanese Patent Laid-Open No. 2002-112095 proposes a method for obtaining two thin-out images between the actually captured images, and calculating the displacement amount between the actually captured images using the thin-out images. Specifically, first, two consecutive shooting operations are carried out while changing the in-focus position in one shutter operation, and an actually captured image A, which is focused on the main object, an actually captured image B, which is focused on the background of the main object, and two thin-out images, which are obtained between the capturing of the actually captured image A and the actually captured image B, are retrieved. The displacement amount of the actually captured image B with respect to the actually captured image A is then calculated using the two thin-out images. The actually captured image A and the actually captured image B, in which the displacement has been corrected based on the calculated displacement amount, are composed to obtain a composite image having a desired defocus degree.
However, in Japanese Patent Laid-Open No. 2002-112095, since there is a temporal difference between the actually captured images and the thin-out images, the displacement amount calculated between the thin-out images is subjected to the influence of hand movement if hand movement occurs between the time of capturing the actually captured images and the time of capturing the thin-out images, for example. Thus, if the displacement amount between the thin-out images is assumed as the displacement amount of the actually captured images, error occurs with respect to the actual displacement amount.
Furthermore, since the displacement amount is calculated using the thin-out images having a smaller size than the actually captured images in Japanese Patent Laid-Open No. 2002-112095, the accuracy of the calculated displacement amount decreases. Specifically, if the displacement amount is calculated with the thin-out image of ⅛ of the actually captured image as in Japanese Patent Laid-Open No. 2002-112095, the displacement amount can only be detected in units of eight pixels when converted to the displacement amount in the actually captured image to be calculated eventually.
Furthermore, since Japanese Patent Laid-Open No. 2002-112095 does not describe a configuration in which the thin-out rate is varied according to the change in the defocus degree between the actually captured images, the process cannot be adaptively changed in accordance with the defocus degree between the actually captured images, and the process of enhancing the alignment accuracy cannot be performed.